{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 音乐网站用户流失预测 -- 特征工程2 & 模型训练 & 预测\n",
    "\n",
    "### 数据集说明\n",
    "\n",
    "项目提供KKBOX用户——歌曲重复播放记录，以及用户和歌曲的元数据。训练数据由2017年2月服务到期的用户构成，target标签代表用户在2017年3月是否续订了业务。测试集中的数据由2017年3月内将到期的用户构成，需要预测用户是否在到期后的一个月内即2017年4月预定、流失的概率。\n",
    "\n",
    "以下是文件及字段说明：\n",
    "\n",
    "1. train.csv: 训练数据，共7,377,418条记录 \n",
    "\n",
    "    msno: 用户id，加密String\n",
    "\n",
    "    song_id: song id，歌曲id\n",
    "\n",
    "    source_system_tab: 触发事件的类型/tab，用于表示app的功能类型\n",
    "\n",
    "    source_screen_name: 用户看到的布局的名字（name of the layout）\n",
    "\n",
    "    source_type: 用户在app上播放音乐的入口的类型\n",
    "\n",
    "    target: 标签。1表示用户在第一次听音乐后会在一个月内继续订阅，0表示没有订阅。\n",
    "\n",
    "2. test.csv ：测试数据，共2,556,790条记录 \n",
    "\n",
    "    id: id (用于结果提交)\n",
    "\n",
    "    msno: 用户id\n",
    "\n",
    "    song_id: 歌曲id\n",
    "\n",
    "    source_system_tab: 触发事件的类型/tab，用于表示app的功能类型\n",
    "\n",
    "    source_screen_name: 用户看到的布局的名字（name of the layout）\n",
    "\n",
    "    source_type: 用户在app上播放音乐的入口的类型\n",
    "\n",
    "3. sampleSubmission.csv：提交结果文件样例 \n",
    "\n",
    "    提交测试结果包含两个字段，分别为测试样本id及其标签为1的概率，格式如下：\n",
    "\n",
    "    id,target\n",
    "    \n",
    "    2,0.3\n",
    "    \n",
    "    5,0.1\n",
    "    \n",
    "    6,1\n",
    "    \n",
    "    etc.\n",
    "\n",
    "4. songs.csv：歌曲元数据信息，用unicode编码 \n",
    "\n",
    "    song_id：歌曲id\n",
    "\n",
    "    song_length: 单位为ms\n",
    "\n",
    "    genre_ids: genre 类别. 可多选，用 “|“隔开\n",
    "\n",
    "    artist_name：歌手\n",
    "\n",
    "    composer：作曲\n",
    "\n",
    "    lyricist：作词\n",
    "\n",
    "    language：语言\n",
    "\n",
    "5. members.csv：用户元数据信息\n",
    "\n",
    "    msno：用户id\n",
    "\n",
    "    city：城市\n",
    "\n",
    "    bd: 年龄。注意：年龄数据有离群点\n",
    "\n",
    "    gender：性别\n",
    "\n",
    "    registered_via: 注册方式\n",
    "\n",
    "    registration_init_time: 注册时间，格式为%Y%m%d\n",
    "\n",
    "    expiration_date: 到期时间，格式为 %Y%m%d\n",
    "\n",
    "6. song_extra_infos.csv：歌曲额外的信息\n",
    "\n",
    "    song_id：歌曲id\n",
    "\n",
    "    song name ：歌曲名字\n",
    "\n",
    "    isrc – 国际标准音像制品编码(International Standard Recording Code )。理论上可用于歌曲id，但产生的ISR没有经过官方授权。因此ISRC中的信息，如国家代码和参考年份可能不正确。且多首歌曲可能共享共一个ISRC，因为一首歌曲的音像制可发行多次。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LGBM_data\n",
      "LR_data\n",
      "members.csv\n",
      "members_copy.csv\n",
      "sample_submission.csv\n",
      "song_extra_info.csv\n",
      "songs.csv\n",
      "submission_lgbm_avg.csv\n",
      "submission_lgbm_avg.csv.gz\n",
      "test.csv\n",
      "train.csv\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/lightgbm/__init__.py:46: UserWarning: Starting from version 2.2.1, the library file in distribution wheels for macOS is built by the Apple Clang (Xcode_8.3.3) compiler.\n",
      "This means that in case of installing LightGBM from PyPI via the ``pip install lightgbm`` command, you don't need to install the gcc compiler anymore.\n",
      "Instead of that, you need to install the OpenMP library, which is required for running LightGBM on the system with the Apple Clang compiler.\n",
      "You can install the OpenMP library by the following command: ``brew install libomp``.\n",
      "  \"You can install the OpenMP library by the following command: ``brew install libomp``.\", UserWarning)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import lightgbm as lgb\n",
    "import datetime\n",
    "import math\n",
    "import gc\n",
    "\n",
    "\n",
    "data_path = '../data/'\n",
    "\n",
    "from subprocess import check_output\n",
    "print(check_output([\"ls\", data_path]).decode(\"utf8\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv(data_path + 'train.csv', dtype={'msno' : 'category',\n",
    "                                                'source_system_tab' : 'category',\n",
    "                                                  'source_screen_name' : 'category',\n",
    "                                                  'source_type' : 'category',\n",
    "                                                  'target' : np.uint8,\n",
    "                                                  'song_id' : 'category'})\n",
    "test = pd.read_csv(data_path + 'test.csv', dtype={'msno' : 'category',\n",
    "                                                'source_system_tab' : 'category',\n",
    "                                                'source_screen_name' : 'category',\n",
    "                                                'source_type' : 'category',\n",
    "                                                'song_id' : 'category'})\n",
    "songs = pd.read_csv(data_path + 'songs.csv',dtype={'genre_ids': 'category',\n",
    "                                                  'language' : 'category',\n",
    "                                                  'artist_name' : 'category',\n",
    "                                                  'composer' : 'category',\n",
    "                                                  'lyricist' : 'category',\n",
    "                                                  'song_id' : 'category'})\n",
    "members = pd.read_csv(data_path + 'members.csv',dtype={'city' : 'category',\n",
    "                                                      'bd' : np.uint8,\n",
    "                                                      'gender' : 'category',\n",
    "                                                      'registered_via' : 'category'},\n",
    "                     parse_dates=['registration_init_time','expiration_date'])\n",
    "songs_extra = pd.read_csv(data_path + 'song_extra_info.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据合并"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train.merge(songs, on='song_id', how='left')\n",
    "test = test.merge(songs, on='song_id', how='left')\n",
    "\n",
    "# 新增一列membership_days表示注册时长\n",
    "members['membership_days'] = members['expiration_date'].subtract(members['registration_init_time']).dt.days.astype(int)\n",
    "\n",
    "# 将注册时间转换为年，月，日\n",
    "members['registration_year'] = members['registration_init_time'].dt.year\n",
    "members['registration_month'] = members['registration_init_time'].dt.month\n",
    "members['registration_day'] = members['registration_init_time'].dt.day\n",
    "\n",
    "# 将到期时间转换为年，月，日\n",
    "members['expiration_year'] = members['expiration_date'].dt.year\n",
    "members['expiration_month'] = members['expiration_date'].dt.month\n",
    "members['expiration_day'] = members['expiration_date'].dt.day\n",
    "\n",
    "# 把日期特征删了\n",
    "members = members.drop(['registration_init_time'], axis=1)\n",
    "members = members.drop(['expiration_date'], axis=1)\n",
    "\n",
    "# 把isrc转换为年\n",
    "def isrc_to_year(isrc):\n",
    "    if type(isrc) == str:\n",
    "        if int(isrc[5:7]) > 17:\n",
    "            return 1900 + int(isrc[5:7])\n",
    "        else:\n",
    "            return 2000 + int(isrc[5:7])\n",
    "    else:\n",
    "        return np.nan\n",
    "        \n",
    "songs_extra['song_year'] = songs_extra['isrc'].apply(isrc_to_year)\n",
    "# 把isrc和name特征删了\n",
    "songs_extra.drop(['isrc', 'name'], axis = 1, inplace = True)\n",
    "\n",
    "# train、test与members、songs_extra合并\n",
    "train = train.merge(members, on='msno', how='left')\n",
    "test = test.merge(members, on='msno', how='left')\n",
    "\n",
    "train = train.merge(songs_extra, on = 'song_id', how = 'left')\n",
    "\n",
    "# train和test对song_length缺失值填充，默认2分钟\n",
    "train.song_length.fillna(200000,inplace=True)\n",
    "train.song_length = train.song_length.astype(np.uint32)\n",
    "train.song_id = train.song_id.astype('category')\n",
    "\n",
    "\n",
    "test = test.merge(songs_extra, on = 'song_id', how = 'left')\n",
    "test.song_length.fillna(200000,inplace=True)\n",
    "test.song_length = test.song_length.astype(np.uint32)\n",
    "test.song_id = test.song_id.astype('category')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 统计genre_id的个数\n",
    "def genre_id_count(x):\n",
    "    if x == 'no_genre_id':\n",
    "        return 0\n",
    "    else:\n",
    "        return x.count('|') + 1\n",
    "# genre_ids的缺失值处理：用no_genre_id填充\n",
    "train['genre_ids'] = train['genre_ids'].cat.add_categories(['no_genre_id']);\n",
    "train['genre_ids'].fillna('no_genre_id', inplace=True);\n",
    "test['genre_ids'] = test['genre_ids'].cat.add_categories(['no_genre_id']);\n",
    "test['genre_ids'].fillna('no_genre_id', inplace=True);\n",
    "# 新增一列genre_ids_count，表示genre_id个数\n",
    "train['genre_ids_count'] = train['genre_ids'].apply(genre_id_count).astype(np.int8)\n",
    "test['genre_ids_count'] = test['genre_ids'].apply(genre_id_count).astype(np.int8)\n",
    "\n",
    "# 统计lyricist个数\n",
    "def lyricist_count(x):\n",
    "    if x == 'no_lyricist':\n",
    "        return 0\n",
    "    else:\n",
    "        return sum(map(x.count, ['|', '/', '\\\\', ';'])) + 1\n",
    "    return sum(map(x.count, ['|', '/', '\\\\', ';']))\n",
    "# lyricist的缺失值处理：用no_lyricist填充\n",
    "train['lyricist'] = train['lyricist'].cat.add_categories(['no_lyricist']);\n",
    "train['lyricist'].fillna('no_lyricist', inplace=True);\n",
    "test['lyricist'] = test['lyricist'].cat.add_categories(['no_lyricist']);\n",
    "test['lyricist'].fillna('no_lyricist', inplace=True);\n",
    "# 新增一列lyricists_count，表示lyricist个数\n",
    "train['lyricists_count'] = train['lyricist'].apply(lyricist_count).astype(np.int8)\n",
    "test['lyricists_count'] = test['lyricist'].apply(lyricist_count).astype(np.int8)\n",
    "\n",
    "# 统计composer个数\n",
    "def composer_count(x):\n",
    "    if x == 'no_composer':\n",
    "        return 0\n",
    "    else:\n",
    "        return sum(map(x.count, ['|', '/', '\\\\', ';'])) + 1\n",
    "# composer的缺失值处理：用no_composer填充\n",
    "train['composer'] = train['composer'].cat.add_categories(['no_composer']);\n",
    "train['composer'].fillna('no_composer', inplace=True);\n",
    "test['composer'] = test['composer'].cat.add_categories(['no_composer']);\n",
    "test['composer'].fillna('no_composer', inplace=True);\n",
    "# 新增一列composer_count，表示composer个数\n",
    "train['composer_count'] = train['composer'].apply(composer_count).astype(np.int8)\n",
    "test['composer_count'] = test['composer'].apply(composer_count).astype(np.int8)\n",
    "\n",
    "# 判断是否有乐队其他助唱\n",
    "def is_featured(x):\n",
    "    if 'feat' in str(x) :\n",
    "        return 1\n",
    "    return 0\n",
    "# artist_name的缺失值处理：用no_artist填充\n",
    "train['artist_name'] = train['artist_name'].cat.add_categories(['no_artist']);\n",
    "train['artist_name'].fillna('no_artist', inplace=True);\n",
    "test['artist_name'] = test['artist_name'].cat.add_categories(['no_artist']);\n",
    "test['artist_name'].fillna('no_artist', inplace=True);\n",
    "# 新增一列is_featured，表示是否有乐队助唱\n",
    "train['is_featured'] = train['artist_name'].apply(is_featured).astype(np.int8)\n",
    "test['is_featured'] = test['artist_name'].apply(is_featured).astype(np.int8)\n",
    "\n",
    "# 统计artist个数\n",
    "def artist_count(x):\n",
    "    if x == 'no_artist':\n",
    "        return 0\n",
    "    else:\n",
    "        return x.count('and') + x.count(',') + x.count('feat') + x.count('&')\n",
    "# 新增一列artist_count，表示artist_name个数\n",
    "train['artist_count'] = train['artist_name'].apply(artist_count).astype(np.int8)\n",
    "test['artist_count'] = test['artist_name'].apply(artist_count).astype(np.int8)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 如果两者相同：artist_name == composer编码为1，否则为0\n",
    "train['artist_composer'] = train.apply(lambda x: 1 if x['artist_name'] == x[\n",
    "    'composer'] else 0,\n",
    "                                       axis=1)\n",
    "test['artist_composer'] = test.apply(lambda x: 1 if x['artist_name'] == x[\n",
    "    'composer'] else 0,\n",
    "                                     axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 如果3者相同： artist, lyricist and composer 编码为1，否则为0\n",
    "train['artist_composer_lyricist'] = train.apply(lambda x: 1 if ((x[\n",
    "    'artist_name'] == x['composer']) & (x['artist_name'] == x['lyricist']) & (\n",
    "        x['composer'] == x['lyricist'])) else 0,\n",
    "                                                axis=1)\n",
    "test['artist_composer_lyricist'] = test.apply(lambda x: 1 if ((x[\n",
    "    'artist_name'] == x['composer']) & (x['artist_name'] == x['lyricist']) & (\n",
    "        x['composer'] == x['lyricist'])) else 0,\n",
    "                                              axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 因为17和45的language出现次数最高，所以新增一列判断是否是这2种语言\n",
    "def song_lang_boolean(x):\n",
    "    if x == x and ('17.0' in str(x) or '45.0' in str(x)) :\n",
    "        return int(1)\n",
    "    return int(0)\n",
    "\n",
    "train['song_lang_boolean'] = train['language'].apply(song_lang_boolean)\n",
    "test['song_lang_boolean'] = test['language'].apply(song_lang_boolean)\n",
    "\n",
    "# 判断是否小于平均时长，如果是，则表明是短音乐，否则是长音乐\n",
    "_mean_song_length = np.mean(train['song_length'])\n",
    "def smaller_song(x):\n",
    "    if x < _mean_song_length:\n",
    "        return 1\n",
    "    return 0\n",
    "\n",
    "train['smaller_song'] = train['song_length'].apply(smaller_song).astype(np.int8)\n",
    "test['smaller_song'] = test['song_length'].apply(smaller_song).astype(np.int8)\n",
    "\n",
    "# 一首歌曲被播放的次数\n",
    "_dict_count_song_played_train = {k: v for k, v in train['song_id'].value_counts().iteritems()}\n",
    "_dict_count_song_played_test = {k: v for k, v in test['song_id'].value_counts().iteritems()}\n",
    "def count_song_played(x):\n",
    "    try:\n",
    "        return _dict_count_song_played_train[x]\n",
    "    except KeyError:\n",
    "        try:\n",
    "            return _dict_count_song_played_test[x]\n",
    "        except KeyError:\n",
    "            return 0\n",
    "    \n",
    "train['count_song_played'] = train['song_id'].apply(count_song_played).astype(np.int64)\n",
    "test['count_song_played'] = test['song_id'].apply(count_song_played).astype(np.int64)\n",
    "\n",
    "# 这个艺术家被演奏了多少次\n",
    "_dict_count_artist_played_train = {k: v for k, v in train['artist_name'].value_counts().iteritems()}\n",
    "_dict_count_artist_played_test = {k: v for k, v in test['artist_name'].value_counts().iteritems()}\n",
    "def count_artist_played(x):\n",
    "    try:\n",
    "        return _dict_count_artist_played_train[x]\n",
    "    except KeyError:\n",
    "        try:\n",
    "            return _dict_count_artist_played_test[x]\n",
    "        except KeyError:\n",
    "            return 0\n",
    "\n",
    "train['count_artist_played'] = train['artist_name'].apply(count_artist_played).astype(np.int64)\n",
    "test['count_artist_played'] = test['artist_name'].apply(count_artist_played).astype(np.int64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将song_lang_boolean转换为int\n",
    "train['song_lang_boolean'] = train['song_lang_boolean'].apply(lambda x : int(float(x)) if x==x else 0)\n",
    "test['song_lang_boolean'] = test['song_lang_boolean'].apply(lambda x : int(float(x)) if x==x else 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>target</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>lyricists_count</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>206471</td>\n",
       "      <td>359</td>\n",
       "      <td>Bastille</td>\n",
       "      <td>Dan Smith| Mark Crew</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>215</td>\n",
       "      <td>1140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>284584</td>\n",
       "      <td>1259</td>\n",
       "      <td>Various Artists</td>\n",
       "      <td>no_composer</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>303617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>225396</td>\n",
       "      <td>1259</td>\n",
       "      <td>Nas</td>\n",
       "      <td>N. Jones、W. Adams、J. Lordan、D. Ingle</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>255512</td>\n",
       "      <td>1019</td>\n",
       "      <td>Soundway</td>\n",
       "      <td>Kwadwo Donkoh</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>187802</td>\n",
       "      <td>1011</td>\n",
       "      <td>Brett Young</td>\n",
       "      <td>Brett Young| Kelly Archer| Justin Ebach</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>412</td>\n",
       "      <td>427</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno  \\\n",
       "0  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "1  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "2  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "3  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "4  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=           explore   \n",
       "1  bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=        my library   \n",
       "2  JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=        my library   \n",
       "3  2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=        my library   \n",
       "4  3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=           explore   \n",
       "\n",
       "    source_screen_name      source_type  target  song_length genre_ids  \\\n",
       "0              Explore  online-playlist       1       206471       359   \n",
       "1  Local playlist more   local-playlist       1       284584      1259   \n",
       "2  Local playlist more   local-playlist       1       225396      1259   \n",
       "3  Local playlist more   local-playlist       1       255512      1019   \n",
       "4              Explore  online-playlist       1       187802      1011   \n",
       "\n",
       "       artist_name                                 composer  ...  \\\n",
       "0         Bastille                     Dan Smith| Mark Crew  ...   \n",
       "1  Various Artists                              no_composer  ...   \n",
       "2              Nas     N. Jones、W. Adams、J. Lordan、D. Ingle  ...   \n",
       "3         Soundway                            Kwadwo Donkoh  ...   \n",
       "4      Brett Young  Brett Young| Kelly Archer| Justin Ebach  ...   \n",
       "\n",
       "  lyricists_count composer_count is_featured  artist_count artist_composer  \\\n",
       "0               0              2           0             0               0   \n",
       "1               0              0           0             0               0   \n",
       "2               0              1           0             0               0   \n",
       "3               0              1           0             0               0   \n",
       "4               0              3           0             0               0   \n",
       "\n",
       "  artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                        0                  0             1   \n",
       "1                        0                  0             0   \n",
       "2                        0                  0             1   \n",
       "3                        0                  0             0   \n",
       "4                        0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  \n",
       "0                215                 1140  \n",
       "1                  1               303617  \n",
       "2                  4                  289  \n",
       "3                  1                    1  \n",
       "4                412                  427  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>lyricists_count</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>224130</td>\n",
       "      <td>458</td>\n",
       "      <td>梁文音 (Rachel Liang)</td>\n",
       "      <td>Qi Zheng Zhang</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>694</td>\n",
       "      <td>13654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>320470</td>\n",
       "      <td>465</td>\n",
       "      <td>林俊傑 (JJ Lin)</td>\n",
       "      <td>林俊傑</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6090</td>\n",
       "      <td>115325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>/uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=</td>\n",
       "      <td>8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=</td>\n",
       "      <td>discover</td>\n",
       "      <td>NaN</td>\n",
       "      <td>song-based-playlist</td>\n",
       "      <td>315899</td>\n",
       "      <td>2022</td>\n",
       "      <td>Yu Takahashi (高橋優)</td>\n",
       "      <td>Yu Takahashi</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>285210</td>\n",
       "      <td>465</td>\n",
       "      <td>U2</td>\n",
       "      <td>The Edge| Adam Clayton| Larry Mullen| Jr.</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "      <td>698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>197590</td>\n",
       "      <td>873</td>\n",
       "      <td>Yoga Mr Sound</td>\n",
       "      <td>Neuromancer</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>180</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                          msno  \\\n",
       "0   0  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "1   1  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "2   2  /uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=   \n",
       "3   3  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "4   4  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=        my library   \n",
       "1  y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=        my library   \n",
       "2  8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=          discover   \n",
       "3  ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=             radio   \n",
       "4  MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=             radio   \n",
       "\n",
       "    source_screen_name          source_type  song_length genre_ids  \\\n",
       "0  Local playlist more        local-library       224130       458   \n",
       "1  Local playlist more        local-library       320470       465   \n",
       "2                  NaN  song-based-playlist       315899      2022   \n",
       "3                Radio                radio       285210       465   \n",
       "4                Radio                radio       197590       873   \n",
       "\n",
       "          artist_name                                   composer  ...  \\\n",
       "0  梁文音 (Rachel Liang)                             Qi Zheng Zhang  ...   \n",
       "1        林俊傑 (JJ Lin)                                        林俊傑  ...   \n",
       "2  Yu Takahashi (高橋優)                               Yu Takahashi  ...   \n",
       "3                  U2  The Edge| Adam Clayton| Larry Mullen| Jr.  ...   \n",
       "4       Yoga Mr Sound                                Neuromancer  ...   \n",
       "\n",
       "  lyricists_count composer_count is_featured  artist_count artist_composer  \\\n",
       "0               0              1           0             0               0   \n",
       "1               2              1           0             0               0   \n",
       "2               1              1           0             0               0   \n",
       "3               0              4           0             0               0   \n",
       "4               0              1           0             0               0   \n",
       "\n",
       "  artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                        0                  0             1   \n",
       "1                        0                  0             0   \n",
       "2                        0                  1             0   \n",
       "3                        0                  0             0   \n",
       "4                        0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  \n",
       "0                694                13654  \n",
       "1               6090               115325  \n",
       "2                  5                  989  \n",
       "3                 31                  698  \n",
       "4                  5                  180  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存特征工程之后的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.to_csv(data_path + 'LGBM_data/fe_train.csv', index=False)\n",
    "test.to_csv(data_path + 'LGBM_data/fe_test.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "========== Train ==========\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>target</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>lyricists_count</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>206471</td>\n",
       "      <td>359</td>\n",
       "      <td>Bastille</td>\n",
       "      <td>Dan Smith| Mark Crew</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>215</td>\n",
       "      <td>1140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>284584</td>\n",
       "      <td>1259</td>\n",
       "      <td>Various Artists</td>\n",
       "      <td>no_composer</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>303617</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>225396</td>\n",
       "      <td>1259</td>\n",
       "      <td>Nas</td>\n",
       "      <td>N. Jones、W. Adams、J. Lordan、D. Ingle</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>255512</td>\n",
       "      <td>1019</td>\n",
       "      <td>Soundway</td>\n",
       "      <td>Kwadwo Donkoh</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>187802</td>\n",
       "      <td>1011</td>\n",
       "      <td>Brett Young</td>\n",
       "      <td>Brett Young| Kelly Archer| Justin Ebach</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>412</td>\n",
       "      <td>427</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno  \\\n",
       "0  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "1  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "2  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "3  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "4  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=           explore   \n",
       "1  bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=        my library   \n",
       "2  JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=        my library   \n",
       "3  2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=        my library   \n",
       "4  3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=           explore   \n",
       "\n",
       "    source_screen_name      source_type  target  song_length genre_ids  \\\n",
       "0              Explore  online-playlist       1       206471       359   \n",
       "1  Local playlist more   local-playlist       1       284584      1259   \n",
       "2  Local playlist more   local-playlist       1       225396      1259   \n",
       "3  Local playlist more   local-playlist       1       255512      1019   \n",
       "4              Explore  online-playlist       1       187802      1011   \n",
       "\n",
       "       artist_name                                 composer  ...  \\\n",
       "0         Bastille                     Dan Smith| Mark Crew  ...   \n",
       "1  Various Artists                              no_composer  ...   \n",
       "2              Nas     N. Jones、W. Adams、J. Lordan、D. Ingle  ...   \n",
       "3         Soundway                            Kwadwo Donkoh  ...   \n",
       "4      Brett Young  Brett Young| Kelly Archer| Justin Ebach  ...   \n",
       "\n",
       "  lyricists_count  composer_count  is_featured  artist_count artist_composer  \\\n",
       "0               0               2            0             0               0   \n",
       "1               0               0            0             0               0   \n",
       "2               0               1            0             0               0   \n",
       "3               0               1            0             0               0   \n",
       "4               0               3            0             0               0   \n",
       "\n",
       "   artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                         0                  0             1   \n",
       "1                         0                  0             0   \n",
       "2                         0                  0             1   \n",
       "3                         0                  0             0   \n",
       "4                         0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  \n",
       "0                215                 1140  \n",
       "1                  1               303617  \n",
       "2                  4                  289  \n",
       "3                  1                    1  \n",
       "4                412                  427  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "========== Test ==========\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>lyricists_count</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>224130</td>\n",
       "      <td>458</td>\n",
       "      <td>梁文音 (Rachel Liang)</td>\n",
       "      <td>Qi Zheng Zhang</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>694</td>\n",
       "      <td>13654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>320470</td>\n",
       "      <td>465</td>\n",
       "      <td>林俊傑 (JJ Lin)</td>\n",
       "      <td>林俊傑</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6090</td>\n",
       "      <td>115325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>/uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=</td>\n",
       "      <td>8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=</td>\n",
       "      <td>discover</td>\n",
       "      <td>NaN</td>\n",
       "      <td>song-based-playlist</td>\n",
       "      <td>315899</td>\n",
       "      <td>2022</td>\n",
       "      <td>Yu Takahashi (高橋優)</td>\n",
       "      <td>Yu Takahashi</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>285210</td>\n",
       "      <td>465</td>\n",
       "      <td>U2</td>\n",
       "      <td>The Edge| Adam Clayton| Larry Mullen| Jr.</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "      <td>698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>197590</td>\n",
       "      <td>873</td>\n",
       "      <td>Yoga Mr Sound</td>\n",
       "      <td>Neuromancer</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>180</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                          msno  \\\n",
       "0   0  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "1   1  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "2   2  /uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=   \n",
       "3   3  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "4   4  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=        my library   \n",
       "1  y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=        my library   \n",
       "2  8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=          discover   \n",
       "3  ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=             radio   \n",
       "4  MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=             radio   \n",
       "\n",
       "    source_screen_name          source_type  song_length genre_ids  \\\n",
       "0  Local playlist more        local-library       224130       458   \n",
       "1  Local playlist more        local-library       320470       465   \n",
       "2                  NaN  song-based-playlist       315899      2022   \n",
       "3                Radio                radio       285210       465   \n",
       "4                Radio                radio       197590       873   \n",
       "\n",
       "          artist_name                                   composer  ...  \\\n",
       "0  梁文音 (Rachel Liang)                             Qi Zheng Zhang  ...   \n",
       "1        林俊傑 (JJ Lin)                                        林俊傑  ...   \n",
       "2  Yu Takahashi (高橋優)                               Yu Takahashi  ...   \n",
       "3                  U2  The Edge| Adam Clayton| Larry Mullen| Jr.  ...   \n",
       "4       Yoga Mr Sound                                Neuromancer  ...   \n",
       "\n",
       "  lyricists_count  composer_count  is_featured  artist_count artist_composer  \\\n",
       "0               0               1            0             0               0   \n",
       "1               2               1            0             0               0   \n",
       "2               1               1            0             0               0   \n",
       "3               0               4            0             0               0   \n",
       "4               0               1            0             0               0   \n",
       "\n",
       "   artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                         0                  0             1   \n",
       "1                         0                  0             0   \n",
       "2                         0                  1             0   \n",
       "3                         0                  0             0   \n",
       "4                         0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  \n",
       "0                694                13654  \n",
       "1               6090               115325  \n",
       "2                  5                  989  \n",
       "3                 31                  698  \n",
       "4                  5                  180  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv(data_path + 'LGBM_data/fe_train.csv')\n",
    "test = pd.read_csv(data_path + 'LGBM_data/fe_test.csv')\n",
    "\n",
    "print(\"=\"*10,\"Train\",\"=\"*10)\n",
    "train.head()\n",
    "print(\"=\"*10,\"Test\",\"=\"*10)\n",
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in train.columns:\n",
    "    if train[col].dtype == object:\n",
    "        train[col] = train[col].astype('category')\n",
    "        test[col] = test[col].astype('category')\n",
    "\n",
    "\n",
    "X_train = train.drop(['target'], axis=1)\n",
    "y_train = train['target'].values\n",
    "\n",
    "X_test = test.drop(['id'], axis=1)\n",
    "ids = test['id'].values\n",
    "\n",
    "# 训练集\n",
    "train_set = lgb.Dataset(X_train, y_train)\n",
    "# 校验集\n",
    "valid_sets = lgb.Dataset(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练1-gbdt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/lightgbm/engine.py:118: UserWarning: Found `num_rounds` in params. Will use it instead of argument\n",
      "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[500]\tvalid_0's auc: 0.833621\tvalid_0's binary_logloss: 0.506766\n",
      "[1000]\tvalid_0's auc: 0.853574\tvalid_0's binary_logloss: 0.481861\n",
      "[1500]\tvalid_0's auc: 0.865212\tvalid_0's binary_logloss: 0.466684\n",
      "[2000]\tvalid_0's auc: 0.874381\tvalid_0's binary_logloss: 0.454273\n",
      "CPU times: user 4h 45min 22s, sys: 18min 53s, total: 5h 4min 15s\n",
      "Wall time: 51min 24s\n"
     ]
    }
   ],
   "source": [
    "params = {\n",
    "        'objective': 'binary',\n",
    "        'boosting': 'gbdt',\n",
    "        'learning_rate': 0.3 ,\n",
    "        'verbose': 0,\n",
    "        'num_leaves': 100,\n",
    "        'bagging_fraction': 0.95,\n",
    "        'bagging_freq': 1,\n",
    "        'bagging_seed': 1,\n",
    "        'feature_fraction': 0.9,\n",
    "        'feature_fraction_seed': 1,\n",
    "        'max_bin': 256,\n",
    "        'max_depth': 10,\n",
    "        'num_rounds': 2000,\n",
    "        'metric' : ['binary_logloss', 'auc']\n",
    "    }\n",
    "\n",
    "%time model_f11 = lgb.train(params, train_set=train_set,  valid_sets=valid_sets, verbose_eval=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<lightgbm.basic.Booster at 0x1831a49b0>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存模型\n",
    "model_f11.save_model(data_path + 'LGBM_data/model_gbdt.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练2-dart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[500]\tvalid_0's auc: 0.813631\tvalid_0's binary_logloss: 0.533959\n",
      "[1000]\tvalid_0's auc: 0.831314\tvalid_0's binary_logloss: 0.513007\n",
      "[1500]\tvalid_0's auc: 0.842779\tvalid_0's binary_logloss: 0.49842\n",
      "[2000]\tvalid_0's auc: 0.852412\tvalid_0's binary_logloss: 0.485436\n",
      "CPU times: user 1d 22h 39min 16s, sys: 48min 54s, total: 1d 23h 28min 11s\n",
      "Wall time: 8h 15min 38s\n"
     ]
    }
   ],
   "source": [
    "params = {\n",
    "        'objective': 'binary',\n",
    "        'boosting': 'dart',\n",
    "        'learning_rate': 0.3 ,\n",
    "        'verbose': 0,\n",
    "        'num_leaves': 100,\n",
    "        'bagging_fraction': 0.95,\n",
    "        'bagging_freq': 1,\n",
    "        'bagging_seed': 1,\n",
    "        'feature_fraction': 0.9,\n",
    "        'feature_fraction_seed': 1,\n",
    "        'max_bin': 256,\n",
    "        'max_depth': 10,\n",
    "        'num_rounds': 2000,\n",
    "        'metric' : ['binary_logloss', 'auc']\n",
    "    }\n",
    "\n",
    "%time model_f21 = lgb.train(params, train_set=train_set,  valid_sets=valid_sets, verbose_eval=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<lightgbm.basic.Booster at 0x1831a6c50>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 保存模型\n",
    "model_f21.save_model(data_path + 'LGBM_data/model_dart.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 单个gbdt结果\n",
    "p_test_1 = model_f11.predict(X_test)\n",
    "# 单个dart结果\n",
    "p_test_2 = model_f21.predict(X_test)\n",
    "# gbdt+dart的平均结果\n",
    "p_test_avg = np.mean([p_test_1, p_test_2], axis = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 生成提交文件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 单个gbdt结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "subm = pd.DataFrame()\n",
    "subm['id'] = ids\n",
    "subm['target'] = p_test_1\n",
    "subm.to_csv(data_path + 'submission_lgbm_gbdt.csv.gz', compression = 'gzip', index=False, float_format = '%.5f')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 单个dart结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "subm = pd.DataFrame()\n",
    "subm['id'] = ids\n",
    "subm['target'] = p_test_2\n",
    "subm.to_csv(data_path + 'submission_lgbm_dart.csv.gz', compression = 'gzip', index=False, float_format = '%.5f')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### gbdt+dart的平均结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "subm = pd.DataFrame()\n",
    "subm['id'] = ids\n",
    "subm['target'] = p_test_avg\n",
    "subm.to_csv(data_path + 'submission_lgbm_avg2.csv.gz', compression = 'gzip', index=False, float_format = '%.5f')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kaggle排名"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**最终Kaggle得分：**\n",
    "\n",
    "- **gbdt的结果：**\n",
    "\n",
    "**得分：0.67834， 排名：374**\n",
    "\n",
    "- **dart的结果：**\n",
    "\n",
    "**得分：0.68596， 排名：271**\n",
    "\n",
    "- **gbdt+dart的平均结果：**\n",
    "\n",
    "**得分：0.68634， 排名：256**\n",
    "\n",
    "**由此可见，平均后的结果最好**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征重要性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>artist_name</td>\n",
       "      <td>23766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>count_song_played</td>\n",
       "      <td>18118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>msno</td>\n",
       "      <td>15638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>count_artist_played</td>\n",
       "      <td>13164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>membership_days</td>\n",
       "      <td>13023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>song_year</td>\n",
       "      <td>10822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>song_length</td>\n",
       "      <td>10731</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>registration_day</td>\n",
       "      <td>10138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>expiration_day</td>\n",
       "      <td>9401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>source_screen_name</td>\n",
       "      <td>7910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>bd</td>\n",
       "      <td>6969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>song_id</td>\n",
       "      <td>6735</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>registration_month</td>\n",
       "      <td>6434</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>composer</td>\n",
       "      <td>5430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>city</td>\n",
       "      <td>5316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>source_type</td>\n",
       "      <td>5231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>genre_ids</td>\n",
       "      <td>4811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>expiration_month</td>\n",
       "      <td>4523</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>lyricist</td>\n",
       "      <td>3156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>language</td>\n",
       "      <td>3151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>registered_via</td>\n",
       "      <td>2571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>composer_count</td>\n",
       "      <td>2108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>registration_year</td>\n",
       "      <td>1988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>gender</td>\n",
       "      <td>1966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>lyricists_count</td>\n",
       "      <td>1533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>source_system_tab</td>\n",
       "      <td>1271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>expiration_year</td>\n",
       "      <td>809</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>song_lang_boolean</td>\n",
       "      <td>406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>genre_ids_count</td>\n",
       "      <td>391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>artist_composer</td>\n",
       "      <td>148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>artist_count</td>\n",
       "      <td>145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>smaller_song</td>\n",
       "      <td>123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>artist_composer_lyricist</td>\n",
       "      <td>69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>is_featured</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        name  importance\n",
       "7                artist_name       23766\n",
       "32         count_song_played       18118\n",
       "0                       msno       15638\n",
       "33       count_artist_played       13164\n",
       "15           membership_days       13023\n",
       "22                 song_year       10822\n",
       "5                song_length       10731\n",
       "18          registration_day       10138\n",
       "21            expiration_day        9401\n",
       "3         source_screen_name        7910\n",
       "12                        bd        6969\n",
       "1                    song_id        6735\n",
       "17        registration_month        6434\n",
       "8                   composer        5430\n",
       "11                      city        5316\n",
       "4                source_type        5231\n",
       "6                  genre_ids        4811\n",
       "20          expiration_month        4523\n",
       "9                   lyricist        3156\n",
       "10                  language        3151\n",
       "14            registered_via        2571\n",
       "25            composer_count        2108\n",
       "16         registration_year        1988\n",
       "13                    gender        1966\n",
       "24           lyricists_count        1533\n",
       "2          source_system_tab        1271\n",
       "19           expiration_year         809\n",
       "30         song_lang_boolean         406\n",
       "23           genre_ids_count         391\n",
       "28           artist_composer         148\n",
       "27              artist_count         145\n",
       "31              smaller_song         123\n",
       "29  artist_composer_lyricist          69\n",
       "26               is_featured           5"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_importance1 = pd.DataFrame({'name':model_f11.feature_name(), 'importance':model_f11.feature_importance()}).sort_values(by='importance', ascending=False)\n",
    "feature_importance1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>artist_name</td>\n",
       "      <td>34961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>msno</td>\n",
       "      <td>33812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>count_song_played</td>\n",
       "      <td>14796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>count_artist_played</td>\n",
       "      <td>12989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>song_year</td>\n",
       "      <td>9358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>song_id</td>\n",
       "      <td>8524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>membership_days</td>\n",
       "      <td>8014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>source_screen_name</td>\n",
       "      <td>6889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>registration_day</td>\n",
       "      <td>6568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>composer</td>\n",
       "      <td>6454</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>expiration_day</td>\n",
       "      <td>6248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>source_type</td>\n",
       "      <td>5467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>song_length</td>\n",
       "      <td>5458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>bd</td>\n",
       "      <td>4619</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>registration_month</td>\n",
       "      <td>4608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>genre_ids</td>\n",
       "      <td>4580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>language</td>\n",
       "      <td>4056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>city</td>\n",
       "      <td>3589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>lyricist</td>\n",
       "      <td>3466</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>expiration_month</td>\n",
       "      <td>3033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>registered_via</td>\n",
       "      <td>1889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>source_system_tab</td>\n",
       "      <td>1467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>gender</td>\n",
       "      <td>1391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>composer_count</td>\n",
       "      <td>1318</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>registration_year</td>\n",
       "      <td>1272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>lyricists_count</td>\n",
       "      <td>899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>expiration_year</td>\n",
       "      <td>852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>song_lang_boolean</td>\n",
       "      <td>629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>genre_ids_count</td>\n",
       "      <td>394</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>artist_count</td>\n",
       "      <td>153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>artist_composer</td>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>smaller_song</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>artist_composer_lyricist</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>is_featured</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        name  importance\n",
       "7                artist_name       34961\n",
       "0                       msno       33812\n",
       "32         count_song_played       14796\n",
       "33       count_artist_played       12989\n",
       "22                 song_year        9358\n",
       "1                    song_id        8524\n",
       "15           membership_days        8014\n",
       "3         source_screen_name        6889\n",
       "18          registration_day        6568\n",
       "8                   composer        6454\n",
       "21            expiration_day        6248\n",
       "4                source_type        5467\n",
       "5                song_length        5458\n",
       "12                        bd        4619\n",
       "17        registration_month        4608\n",
       "6                  genre_ids        4580\n",
       "10                  language        4056\n",
       "11                      city        3589\n",
       "9                   lyricist        3466\n",
       "20          expiration_month        3033\n",
       "14            registered_via        1889\n",
       "2          source_system_tab        1467\n",
       "13                    gender        1391\n",
       "25            composer_count        1318\n",
       "16         registration_year        1272\n",
       "24           lyricists_count         899\n",
       "19           expiration_year         852\n",
       "30         song_lang_boolean         629\n",
       "23           genre_ids_count         394\n",
       "27              artist_count         153\n",
       "28           artist_composer         121\n",
       "31              smaller_song          64\n",
       "29  artist_composer_lyricist          49\n",
       "26               is_featured          13"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_importance2 = pd.DataFrame({'name':model_f21.feature_name(), 'importance':model_f21.feature_importance()}).sort_values(by='importance', ascending=False)\n",
    "feature_importance2\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
